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Background / Context: 

Description of prior research and its intellectual context. 

U.S. Education Secretary Arne Duncan argued in a 2010 address to the College Board, 
"High schools must shift from being last stop destinations for students on their education 
journey to being launching pads for further growth and lifelong learning for all students. The 
mission of high schools can no longer be to simply get students to graduate. Their expanded 
mission... must also be to ready students for careers and college." What will it take to meet this 
challenge? Senior year in the Chicago Public Schools (CPS) presents an interesting dilemma in 
this changing educational landscape. While all CPS graduates take what might be termed a 
college preparatory curriculum in order to fulfill graduation requirements, those requirements 
can largely be fulfilled by junior year. This leaves senior year as the only year in high school with 
little guidance about what courses students should take - and also leaves space for a great deal 
of variation in students' enrollment in advanced coursework like a fourth year of math or an AP 
course. These senior year coursework decisions could be critical for college access and success. 
Advanced coursework may be critical in boosting students' academic skills for college, and it 
could also be the case that not taking advanced coursework may make college- bound students 
less desirable applicants in an increasingly competitive college admissions pool. Senior year 
may be a critical lever in addressing college readiness and performance, and we must build an 
understanding of what matters for which students in shaping postsecondary outcomes 

There is already a moderate body of research on this topic that addresses various 
aspects of these questions. There are a number of studies that support the hypothesis that AP 
courses are able to increase student access to college (Roderick and Stoker 2010, Jackson 2007, 
Long and Conger 2012). Other studies with rigorous methods find that students who take 
higher level math courses are less likely to be placed in remedial education, are more likely to 
receive higher grades in college and are more likely to persist and graduate (Joensen & Neilsen 
2009, Rose and Betts 2004). Finally, the well known Answers in the Toolbox (Adelman 1999) 
established a link between the rigor of student's coursework, high school G PA, and test scores 
and the likelihood of college graduation. While all these studies have focused on the 
relationship between high school courses and later educational outcomes, they do not focus 
specifically on senior year. 


Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This study has taken an in-depth look at senior year coursetaking patterns in CPS. Using both 
quantitative and qualitative methods we examine both the effects senior year coursetaking has 
on college outcomes and student reports of their experiences in senior year. Specifically, we 
quantitatively ask what effect AP courses, a fourth year of math, or a high concentration of core 
subjects in a student's senior year schedule can have on enrolling in a four-year college, 
enrolling in a more selective four-year college, and persisting in a four year college for two 
years. Qualitatively, we examine the level of challenge reported by students in individual 
classes as well as across senior year as a while. All of our analysis is run across separate 
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achievement subgroups in order to observe any heterogeneity of treatment effects based on 
student achievement. 


Setting: 

Description of the research location. 

This study focuses exclusively on CPS seniors. CPS is the third largest school system in 
the country, with 122 high schools, and approximately 400,000 students. In Chicago, Illinois, 
and across most of the nation, high school curricula and course offerings have shifted over the 
past twenty years to align with college readiness standards. Chicago has seen a massive 
expansion of AP courses offered, along with a change in graduation requirements to align with 
what colleges broadly say is required to gain admittance, effectively driving an increase in core 
coursetaking over time but leaving little guidance for senior year. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The quantitative portion of this study focused on 30,649 CPS graduates from the 
graduating cohorts of 2003-2007. These were students who had access to either a somewhat 
selective or selective college based on their ACT scores and cumulative GPAs. Students with 
higher or lower achievement did not exhibit enough variation in coursetaking patterns to 
reliably estimate an impact. Of these graduates, 15% were white, 41% were African American, 
35% were Latino and 8% were Asian; 64% of these graduates were female. Approximately half 
of these students took an advanced math course (Pre-Calculus, Calculus, AP Statistics or 
Computer Programming), 37% of students took at least one AP course, and 63 % took at least 
four core academic subjects their senior year. The qualitative portion of this study focuses on 
93 students from the graduating class of 2006. The sample is gender-balanced and reflects the 
racial/ethnic composition of CPS students across the city. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

We were able to identify, from the literature, three key indicators of a strong senior 
year schedule: four or more core academic subject classes, a fourth year of math, or at least 
one AP course. Participation in these courses is stratified by achievement across the district, but 
more importantly, it is also stratified within achievement categories across high schools. For 
example, students with access to a somewhat selective college (i.e. those with grades and ACT 
scores that would likely give them access to most public colleges in Illinois) participate in 
fourth-year math at rates as low as 20 percent and as high as 90 percent across high schools 
(See Figure 1). This variation within achievement groups exists for participation in AP and four 
or more core classes as well. 

Research Design: 

Description of the research design. 

In any investigation involving student choice, self selection is a major source of bias. We 
use a combination of propensity scores and instrumental variables to address the selection 
problem. First, we separately model the propensity of each indicator in schools where many 
students took these courses. Then using this model, we simulate the propensity for taking these 
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courses in schools where few students take these courses because of administrative decisions 
about course offerings. A key condition for instrumental variables to be valid is that differences 
in exposure to these courses are only related to the outcomes of interest through the 
treatment rather than exogenous factors like student motivation. This condition would be 
violated if students who attended schools that offer high rates of fourth year math or AP 
classes differ systematically from schools that have fewer students taking these classes. 
However, we find that there is significant overlap in respect to school-wide GPAs and ACT 
scores, and we account for further marginal differences within the propensity model. Our 
qualitative study design required intentional selection of both high schools and students to 
examine variation in coursetaking. The high schools selected for participation in the study were 
schools where students had participated in a range of curricular tracks in high school (IB, 
AP/honors, regular, vocational, etc.), and we slightly oversampled students in advanced 
curricular tracks in order to better understand the experiences students had in advanced 
coursework. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

This study uses administrative data from the Chicago Public Schools, archived by the 
Consortium on Chicago School Research at the University of Chicago. The current study uses 
data on students' backgrounds, test scores, and course taking patterns, as well as school 
compositional data. In order to identify whether graduates enroll in college in the fall after 
graduation, the kinds of colleges they attend, and their two-year persistence rates, we use data 
from the National Student Clearinghouse (NSC). Currently, NSC's enrollment verification 
program covers 92 percent of postsecondary enrollment in the United States. 

Data was analyzed using the combined propensity score and instrumental variable 
approach described above. First, we estimate the probability of taking a certain course for 
students at schools that have high incidences of offering them, conditional on students' 
subject-specific 11th grade GPAs, ACT scores, interactions between the two, indicators about 
past course taking, and demographic information. Using the estimated model, we then simulate 
the propensity for taking these courses among students who attend schools that do not offer 
these courses, essentially simulating what would have happened if these students had gone to 
schools that offered AP math and science courses. Finally we create a sample of matched pairs 
based on these propensity scores to run our outcome models for each college outcome. This 
analysis was run for each achievement group to avoid heterogeneous comparisons. 

In our qualitative study, students were interviewed three times during their senior year 
and were asked questions about their courses as part of a larger study on the transition to 
college. Researchers read each case in its entirety and assigned ratings for the level of challenge 
of a student's senior year overall, as well as for each class for which there was sufficient data to 
assign an individual qualitative code. See Table 3 for a list of qualitative codes. 

Findings / Results: 

Description of the main findings with specific details. 

Once we control for selection, we find modest effects of senior year course taking on 
college outcomes. For students with access to somewhat selective colleges (most public 
colleges in Illinois), we find that taking at least one AP course, or taking a fourth year of math. 
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improves the likelihood of attending a four year college. For students with access to selective 
colleges, taking at least two AP courses, in any subjects, improves the likelihood of attending a 
more selective college. For all groups of students, we find no effects of senior year course 
taking on college persistence (see Tables 1 and 2 for full results from analysis). Our qualitative 
investigation found that, with the exception of those in the IB program, students 
overwhelmingly described senior year as a dismally unchallenging experience: easier than 
previous years of high school and requiring little work effort. Sub-groups analysis indicated that 
even students who took rigorous-looking schedules -(AP, math, etc.) were still unlikely to 
describe senior year as challenging. A further analysis of students' reports of challenge in 
individual classes across course types revealed that students did indeed find their AP courses to 
be challenging. Their senior year schedules, however, were filled with unchallenging elective 
courses, which dominated their perception of senior year overall. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

This study leads to a number of seemingly contradictory findings. First, although there 
are some courses, like AP, that help students gain access to college, they do not seem to have 
any effect on persistence. Second, though AP courses stood out as distinctly challenging to 
students, there is again no evidence of long term positive effects for these classes. Lastly, it is 
surprising that even students who took challenging advanced courses still typically described 
senior year as unchallenging. These findings make more sense, however, in the context of a 
student's overall academic experience. While advanced courses like AP might be useful tools for 
the college transition, one or two effective courses is not the same thing as an intentionally 
designed, day-long, year-long college preparatory experience. The promise and limits of adding 
individual advanced courses to a student's senior year schedule presents a complicated story 
for policymakers who are concerned both with access and attainment in postsecondary 
education. Would requiring these courses help students in the long term or simply add another 
barrier to high school graduation? Is expanding access to AP coursework worth the cost? Is it 
important for college-bound seniors to take a math course? The analysis in this study suggests 
that the answers to these questions are not straightforward, and that ultimately, students' 
postsecondary success will have as much to do with what happens outside of these courses as 
within them. 
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Figure 1 : Participation in advanced math courses varies widely across high 
schools, even among similarly qualified students 



Note: Numbers in this table come from 2003-2009 CPS graduates who were not enrolled in special education and not 
in alternative or charter high schools. Data points were only included if school had 15 or more students in a given 
category. 


Table 1: Taking AP courses and a fourth year of math significantly increases the 
probability that CPS students positioned to enroll in somewhat selective colleges 
will enroll in a four-year college. 


Not addressing selection leads to overestimates of the benefits of coursetaking 




Enrolling in a four year college 

Enrolling in selective or very 
selective among four-year college 
goers 

2 year retention within a four- 
year college 



Predicted probability 

Predicted probability 

Predicted probability 



No selection 

With selection 

No selection 

With selection 

No selection 

With selection 



controls 

control 

controls 

controls 

controls 

controls 


NoAP 

42 

41.8 

14 

13.8 

61 

61.2 

AP 

One AP 

50.5 

50.4 

17.8 

15.8 

65.1 

54.0 


Two or more AP 

54 

51.0 

21.1 

14.3 

77.2 

52.7 

Effect 

One AP 

8.5** 

8.5** 

3.8* 

Not Sig 

5.1** 

Not Sig 

(difference 
from no AP) 

2 or more AP 

12 ** 

9.1** 

7.1** 

Not Sig 

16.2** 

Not Sig 

Fourth year 

No math 

42 

40.5 

13 

13.1 

55 

55.4 

math 

Math 

46.6 

44.7 

15.3 

16.2 

56.7 

55.8 

Effect of math 


4.6* 

4.1** 

Not Sig 

Not Sig 

1.7* 

Not Sig 

Four or more 

< 4 core classes 

44 

43.7 

10 

9.7 

55 

55.1 

core 

4 or more core 
classes 

46.3 

44.8 

12.3 

14.9 

28.4 

55.8 

Effect of 4 
core 


2.3* 

Not sig 

Not Sig 

5.2** 

3.5* 

Not Sig 
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Table 2: Taking two or more AP courses significantly increases the probability that 
CPS students on pace to selective colleges will enroll in a more selective college. 

Not addressing selection leads to overestimates of the benefits of coursetaking 




Enrolling in a 

four year college 

Enrolling in selective or very 
selective among four year 
college goers 

2 year retention within a four 
year college 



Predicted probability 

Predicted probability 

Predicted probability 



No selection 

With selection 

No selection 

With selection 

No selection 

With selection 



controls 

control 

controls 

controls 

controls 

controls 

Advanced 

NoAP 

67 

66.6 

33 

33.0 

70 

70.3 

placement 

One AP 

72.8 

71.7 

41.2 

36.4 

73.8 

68.3 


Two or more AP 

77.5 

71.2 

53 

51.7 

77.2 

77.2 

Effect 

One AP 

5.8** 

Not Sig 

8.2** 

Not Sig 

3.8** 

Not Sig 

(difference 
from no AP) 

2 or more AP 

10.5** 

Not Sig 

20.0** 

18.7** 

7.2** 

Not Sig 

Fourth year 

No math 

61 

61.1 

35 

35.5 

71 

73.5 

math 

Math 

67.9 

62.0 

35.8 

38.7 

75.1 

72.1 

Effect of math 


7.0** 

Not Sig 

Not Sig 

Not Sig 

Not Sig 

Not Sig 

Four or more 

< 4 core classes 

69 

68.8 

35 

35.3 

72 

71.2 

core 

4 or more core 
classes 

73.8 

71.9 

38.3 

36.9 

69 

72.3 

Effect of 4 core 


4.8** 

Not Sig 

3.3** 

Not Sig 

Not Sig 

Not Sig 
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Table 3: 

Coding for Senior Year Challenge 


Question 

Responses 

Code 

Considered 

Senior Year Overall 

Is senior year harder or easier 
than previous years of high 
school? 

Easier or the same 

Low 

Harder 

High 

How much homework are you 
doing this year? Is that more or 
less than last year? 

No homework or virtually no homework; less homework than 
last year. 

Low 

Some homework (approximately weekly); the same as last year 

Medium 

Consistent homework (approximately daily); more homework 
than last year 

High 

Is your homework difficult or 
challenging to complete? Is it any 
harder or easier than last year? 

Not challenging; less challenging than last year 

Low 

Neither challenging nor easy; the same as last year 

Medium 

Challenging; more challenging than last year 

High 

Are you learning a lot in your 
classes this year? 

No— Not learning anything; learning less than last year 

Low 

Yes— Learning college-oriented skills; learning more than last 
year 

High 

If 1 offered you the choice to go 
back and skip senior year, and you 
could go straight from junior year 
to college, would you do it? 

Yes— Nothing important learned this year 

Low 

No— Learned important things this year 

High 


Individual Classes Senior Year 

Do you have a lot of homework 
for this class? How often is 
homework assigned? How long 
does it take to complete? How 
much of it do you do during 
class time? 

No homework or virtually no homework at all; any homework 
assigned can be finished during the class period 

Low 

Some homework (approximately weekly); at least some 
homework necessary to complete at home 

Medium 

Daily homework; projects with clear deadlines; completion of 
homework required to adequately prepare for class 

High 

What is that class like? How do 
you spend your time in that class? 
What happens on a typical day in 
that class? 

Class is slow moving; class requires little to no participation or 
attention from student; student is accountable for little of the 
information imparted in class 

Low 

Student is busy at least part of the time in class; information 
imparted in class is somewhat important (i.e.. student takes 
notes on lectures to use for studying) 

Medium 

Student is working productively the entire class period; student 
is held accountable for information imparted in class 

High 

Is this class any easier or harder 
than the classes you've taken 
in the same subject in previous 
years? 

Easier 

Low 

The same 

Medium 

Harder 

High 

Is the work for this class— either 
during class or work you take 
home— challenging to complete? 

Work is easy or can consistently be done at the last minute; 
work is the same thing the student did last year; doesn't matter 
if you miss a day of class 

Low 

Work is neither hard nor easy; work is harder but can't describe 
why; student has to participate in class to some extent 

Medium 

Work is hard, and student can describe why: student really 
has to pay attention in class and/or to the homework in order 
to understand the concepts; class work or homework requires 
student to think deeply or in new ways; student believes missing 
one day of class is risky 

High 

Are you learning a lot in this 
class? 

No 

Low 

Yes 

High 


SREE Spring 2013 Conference Abstract Template 


B-3 








